1
00:00:00,510 --> 00:00:00,870
All right.

2
00:00:00,870 --> 00:00:02,250
Welcome back.

3
00:00:02,250 --> 00:00:09,450
This is a two part lesson again and this lesson we will introduce open CV and start working on doing

4
00:00:09,450 --> 00:00:14,050
our first part of the data processing by the end of the lesson.

5
00:00:14,070 --> 00:00:17,390
We will have done the first five steps of our roadmap.

6
00:00:17,400 --> 00:00:18,970
We're going to get up to here.

7
00:00:19,530 --> 00:00:22,890
Well we've got a cropped image.

8
00:00:22,890 --> 00:00:29,700
So as you can see I've got my Python server running on port eight thousand and I've got my local host

9
00:00:29,760 --> 00:00:34,490
up here in Google Chrome with the developer console on the side.

10
00:00:34,500 --> 00:00:40,680
The first thing we're gonna do is include the open see the library in our project and we've seen one

11
00:00:40,680 --> 00:00:48,450
way of doing it which is using a C.D. And here with tensor flow jazz and all we had to do for that was

12
00:00:48,540 --> 00:00:50,970
include this link.

13
00:00:50,970 --> 00:00:56,250
Now I want to show you a different way of including a third party library and it's gonna be more similar

14
00:00:56,250 --> 00:01:01,230
to how we're including our drawing that J S file right here.

15
00:01:01,230 --> 00:01:06,240
So the first thing I'm going to do is I want to right click and create a new folder in my Explorer pane

16
00:01:07,080 --> 00:01:14,760
and I'm going to call this folder vendor and then what I'm gonna do is I'm going to go and download

17
00:01:15,120 --> 00:01:23,820
the open see the javascript file you can find this file under docks thought open C dot org and here's

18
00:01:23,820 --> 00:01:24,960
the version number.

19
00:01:24,960 --> 00:01:32,280
And this is the name of the javascript file open CV dot J S I want to put this link into the course

20
00:01:32,280 --> 00:01:40,560
resources of course but the important thing to do is to save this thing as open CV dot J.S.

21
00:01:43,260 --> 00:01:49,890
once you've saved the file in your Downloads folder all you need to do is need to drag it into your

22
00:01:49,890 --> 00:01:51,020
project folder.

23
00:01:51,270 --> 00:01:57,330
So that's where we credit this vendor folder just a moment ago and we're just gonna drag and drop it

24
00:01:57,330 --> 00:01:57,540
in.

25
00:01:58,320 --> 00:02:05,970
Alternatively you can even drag and drop it directly into the explorer pain and the s code that should

26
00:02:06,000 --> 00:02:13,270
also add it right here in Windows Explorer or in Mac finder so now that we've heard of this entire library

27
00:02:13,270 --> 00:02:22,180
of code and you can see it's quite a bit to our project all we need to do is add a script tag once again

28
00:02:22,600 --> 00:02:24,940
to be able to use our library of code.

29
00:02:25,480 --> 00:02:34,180
So once again it's a script tag that has a source and the source is going to be the relative path to

30
00:02:34,510 --> 00:02:41,710
our open CV file and the relative path is going to be under vendor and then open CBD at J.S..

31
00:02:41,860 --> 00:02:45,310
And you can see the autocomplete here suggesting the file already.

32
00:02:45,310 --> 00:02:47,280
So I don't even have to type this out.

33
00:02:47,650 --> 00:02:54,580
Now don't forget to close the script tag and we're ready to go the next thing we're gonna do is we're

34
00:02:54,580 --> 00:03:00,790
gonna split out the logic where we pre process our data from our index thought each to Mal file and

35
00:03:00,790 --> 00:03:04,480
from the drawing that j ust file where we've got our drawing logic.

36
00:03:05,110 --> 00:03:10,330
So we're gonna right click and create a new file in our root directory and we're just gonna call this

37
00:03:10,720 --> 00:03:17,650
processing dot J.S. and the first thing we're going to do is we're going to take the code that we've

38
00:03:17,650 --> 00:03:23,380
written here in our index that each terminal where we've got our function where we load our model and

39
00:03:23,380 --> 00:03:28,910
we're going to move it over by cutting it and pasting it into processing.

40
00:03:29,020 --> 00:03:32,320
J.S. so let's save this file.

41
00:03:32,460 --> 00:03:42,670
Let's save our index that HMO file and then let's go into chrome and refresh our site now.

42
00:03:42,750 --> 00:03:47,310
What you should see here in the developer console is an error.

43
00:03:47,460 --> 00:03:53,810
You should see an error that reads uncorked reference error load model is not defined.

44
00:03:53,820 --> 00:03:55,730
Now I want to pose a challenge to you.

45
00:03:55,890 --> 00:03:59,130
Can you diagnose and fix this error.

46
00:03:59,130 --> 00:04:06,910
I want to give you a few seconds to pause the video and give this a go.

47
00:04:06,970 --> 00:04:07,750
Ready.

48
00:04:07,750 --> 00:04:09,230
Here's the solution.

49
00:04:09,340 --> 00:04:16,690
The hint the error message is giving me is that the problem is on line number forty five and looking

50
00:04:16,810 --> 00:04:23,890
at our vice code I can see that on line forty five is this load model function so load model is not

51
00:04:23,890 --> 00:04:31,150
defined what that means is that our website doesn't know what load model means even though we've defined

52
00:04:31,150 --> 00:04:35,210
it right here in the processing Doc J S file.

53
00:04:35,800 --> 00:04:41,710
The reason we're getting this error is because our index that each team alpha cannot talk to our processing

54
00:04:41,710 --> 00:04:43,120
file yet.

55
00:04:43,390 --> 00:04:48,800
The part that we've missed out on is adding a script tag to link the two.

56
00:04:49,060 --> 00:04:57,250
So we can do that right below where we've added our drawing not J S file so script is our c is equal

57
00:04:57,250 --> 00:05:03,860
to and then it's processing Doc J.S. There you go.

58
00:05:03,890 --> 00:05:11,450
If I close the script tag save come back into chrome refresh then what we should see is that era.

59
00:05:11,450 --> 00:05:12,140
Go away.

60
00:05:12,980 --> 00:05:14,180
Fantastic.

61
00:05:14,180 --> 00:05:20,690
So now that we've got this working let's add a new function here I want to minimize the load model function

62
00:05:21,440 --> 00:05:24,670
and then I'm going to format this document here.

63
00:05:24,710 --> 00:05:30,200
It's gonna right click and click on format document that cleans up my indentation.

64
00:05:30,470 --> 00:05:35,950
And in order to add the function I want to use the snippet here from this code.

65
00:05:36,050 --> 00:05:41,810
So the function that I'm gonna create is gonna be called predict image.

66
00:05:41,810 --> 00:05:44,300
That's what we're gonna do our pre processing.

67
00:05:44,450 --> 00:05:45,940
And it's not going to take any inputs.

68
00:05:45,950 --> 00:05:50,110
So I want to delete the parameters here inside predict image.

69
00:05:50,990 --> 00:05:55,950
I'm just gonna add a log statement first and it's gonna read processing.

70
00:05:55,950 --> 00:06:01,640
Dot dot dot dot and try and call this function when we press the button.

71
00:06:01,640 --> 00:06:08,120
So inside the index that each time l file we're not just going to clear our canvas but before we clear

72
00:06:08,120 --> 00:06:13,610
the canvas we're gonna call this predict image function.

73
00:06:13,610 --> 00:06:14,960
There we go.

74
00:06:15,020 --> 00:06:16,750
So let's save our changes.

75
00:06:16,820 --> 00:06:22,480
Come back here and let's try and refresh and see if we can spot that log statement.

76
00:06:22,520 --> 00:06:22,790
All right.

77
00:06:23,000 --> 00:06:28,770
So when I press the button and I get an error predict image is not defined.

78
00:06:28,850 --> 00:06:30,430
So we've seen this before right.

79
00:06:30,500 --> 00:06:32,120
What's going on.

80
00:06:32,120 --> 00:06:34,640
Do we have the same problem as before.

81
00:06:34,640 --> 00:06:37,930
Well no because we've added the script tag right here.

82
00:06:38,150 --> 00:06:43,240
So the index that each team l file and the processing that just file should be linked.

83
00:06:43,280 --> 00:06:44,060
Right.

84
00:06:44,210 --> 00:06:50,560
The spelling of our function name here also matches the spelling of our function name here.

85
00:06:50,720 --> 00:06:56,190
The reason I'm getting this error is because I've got unsafe changes in my processing that J S file.

86
00:06:56,300 --> 00:06:59,690
That's why it doesn't recognize this function.

87
00:06:59,690 --> 00:07:08,870
So once I saved this file and come back here and refresh and click the button I've got my log statement

88
00:07:09,050 --> 00:07:11,180
showing up in the console.

89
00:07:11,180 --> 00:07:16,670
Now that we've got that working let's take the next step namely reading and being able to display an

90
00:07:16,670 --> 00:07:19,130
image using open CV.

91
00:07:19,550 --> 00:07:26,180
So what we want to do is if we draw something on the canvas we'll be able to load this image into open

92
00:07:26,180 --> 00:07:27,120
CV.

93
00:07:27,320 --> 00:07:31,710
The key to getting this working is to have a look at the documentation.

94
00:07:31,790 --> 00:07:36,910
So on this page right here you'll see that there are really two key methods.

95
00:07:36,980 --> 00:07:40,210
The first method is called I Am read.

96
00:07:40,460 --> 00:07:45,280
And the second method for displaying an image is called I Am show.

97
00:07:45,290 --> 00:07:52,670
So what we're gonna do is we're gonna use I am read to read our image from our canvas scrolling down

98
00:07:52,700 --> 00:07:54,120
in the documentation.

99
00:07:54,200 --> 00:07:56,920
We can even see this code snippet right here.

100
00:07:57,410 --> 00:08:02,240
So it says let's image equals c v dot I am read.

101
00:08:02,240 --> 00:08:05,200
And then in the parentheses it has image source.

102
00:08:05,270 --> 00:08:10,180
This is the format of the code that we're gonna be using inside our predict image function.

103
00:08:10,220 --> 00:08:11,880
Let's add this very line.

104
00:08:11,930 --> 00:08:15,950
Let's image as equal to c v dot.

105
00:08:16,220 --> 00:08:18,060
I am read.

106
00:08:18,290 --> 00:08:24,230
And then in the parentheses we're going to add the image source and the source for us is our canvas

107
00:08:25,250 --> 00:08:26,120
and that's it.

108
00:08:26,120 --> 00:08:28,260
That's how we load an image.

109
00:08:28,520 --> 00:08:34,490
I think to understand with this line of code is first off we're creating a variable and then we're giving

110
00:08:34,490 --> 00:08:36,990
that variable a value here.

111
00:08:37,010 --> 00:08:40,090
This let keyword is one that we haven't seen before.

112
00:08:40,340 --> 00:08:48,170
But it's very very similar to const or var The only difference is that lat allows us to create this

113
00:08:48,320 --> 00:08:56,090
image variable locally within this predict image function unlike the canvas which is a global variable.

114
00:08:56,090 --> 00:09:03,290
We've created it here in our drawing that J S file and we can access it anywhere in this file and we

115
00:09:03,290 --> 00:09:10,400
can even access it inside the processing dot j ust file image in contrast is a local variable meaning

116
00:09:10,400 --> 00:09:18,710
it only exists inside the predict image function that's what that led keyword is saying for more information

117
00:09:18,860 --> 00:09:27,050
about the differences between let const and var I would recommend heading to this page here from W three

118
00:09:27,050 --> 00:09:35,370
schools on this page you can learn all about the three key words let const and var and the differences

119
00:09:35,390 --> 00:09:42,500
that they imply for the scope of the variable so with var we've often got global scope but there's something

120
00:09:42,500 --> 00:09:50,960
like led we have something called block scope CV is our open CV library and we're accessing our I am

121
00:09:50,960 --> 00:09:59,270
read function from the open CV library that's what this line of code means in a nutshell but what I

122
00:09:59,270 --> 00:10:06,410
reckon is that we should really verify that we've successfully loaded our image so let's create a second

123
00:10:06,410 --> 00:10:14,090
canvas on our website where we can preview our output so what I want to do is I want to create some

124
00:10:14,180 --> 00:10:20,180
canvases just below this check and button that preview what it is that our code is doing.

125
00:10:20,600 --> 00:10:26,690
The way to create a canvas is outlined here on this page here from W3 schools.

126
00:10:26,990 --> 00:10:35,310
So previously what we've done is we've used get element by I.D. to pick out a canvas from our age html

127
00:10:35,320 --> 00:10:43,010
file but we can actually also just create a canvas object using JavaScript straight up and we can do

128
00:10:43,010 --> 00:10:49,700
that with document don't create element and then canvas in the parentheses.

129
00:10:49,810 --> 00:10:54,650
So let's try this out when I add a little comment here and it's gonna read.

130
00:10:54,670 --> 00:11:00,100
Testing only meaning we're actually going to delete this later.

131
00:11:00,100 --> 00:11:08,470
And here what we're gonna do is we're gonna create a constant called let's call it output canvas and

132
00:11:08,740 --> 00:11:16,810
that is going to hold on to our canvas element that we're going to create using document don't create

133
00:11:16,870 --> 00:11:17,650
element.

134
00:11:17,890 --> 00:11:22,420
And then in the parentheses it's going to be single quotes and canvas.

135
00:11:22,510 --> 00:11:30,550
Now that we've got our testing canvas we can use CV dot I am show in order to output what it is that

136
00:11:30,550 --> 00:11:30,970
we've got.

137
00:11:31,450 --> 00:11:33,970
So CV dot I am show.

138
00:11:33,970 --> 00:11:36,620
And then when we put between the two parentheses.

139
00:11:37,150 --> 00:11:43,900
Well for that we should check the documentation once again I am show only needs two things.

140
00:11:44,050 --> 00:11:52,260
It needs a canvas output so it needs to know where to display the image and it needs the image itself.

141
00:11:52,270 --> 00:11:59,810
So the one that we read from the image source so let's add that between the two parentheses.

142
00:11:59,810 --> 00:12:03,290
The place we want the image to show up is our output canvas.

143
00:12:03,290 --> 00:12:07,910
And of course our image itself is actually called image.

144
00:12:07,910 --> 00:12:10,400
That's this one right here.

145
00:12:10,400 --> 00:12:18,140
Now that we've got that we can display it on our website and we're gonna do that simply by appending

146
00:12:18,560 --> 00:12:22,340
our output canvas to the body of our document.

147
00:12:22,340 --> 00:12:25,420
So the code for that looks like this document.

148
00:12:25,460 --> 00:12:32,930
What body don't append child so the canvas that we're testing here is just gonna be appended to the

149
00:12:32,930 --> 00:12:36,100
body using the append child method.

150
00:12:36,140 --> 00:12:40,050
So now that we've got our testing code finished let's try it out.

151
00:12:40,300 --> 00:12:43,490
What I want to do here is going to zoom out a little bit.

152
00:12:43,760 --> 00:12:50,420
So I'm just going to make the website slightly smaller and I want to draw something in my canvas.

153
00:12:50,660 --> 00:12:51,860
Click check answer.

154
00:12:52,310 --> 00:12:54,680
And lo and behold the canvas.

155
00:12:54,770 --> 00:13:01,940
The second one for testing appears below the button and I can draw again and click and you should see

156
00:13:01,940 --> 00:13:09,230
that you're able to generate these testing canvases side by side as they're appended to the body of

157
00:13:09,260 --> 00:13:10,860
the document.

158
00:13:10,880 --> 00:13:11,140
All right.

159
00:13:11,170 --> 00:13:12,770
So where does this leave us.

160
00:13:12,770 --> 00:13:20,990
Well we've successfully managed to load our image the next step is converting the image to black and

161
00:13:20,990 --> 00:13:25,440
white and to a single colour channel.

162
00:13:25,460 --> 00:13:32,360
Now I know that on our Web site it already looks like we're dealing with a black and white image but

163
00:13:32,570 --> 00:13:39,230
this image is actually still in our GP format with three colour channels not in grayscale with one colour

164
00:13:39,230 --> 00:13:41,230
channel which is what we need.

165
00:13:41,240 --> 00:13:44,220
So how do we convert this to grayscale.

166
00:13:44,240 --> 00:13:47,730
Well the open CV documentation is our friend.

167
00:13:47,750 --> 00:13:54,710
Once again what we see on this page here which is entitled changing colour spaces as part of the open

168
00:13:54,710 --> 00:14:03,710
CV tutorial on javascript we see that there is this function here called convert colour CBT colour and

169
00:14:03,800 --> 00:14:10,260
if we scroll down a little bit we see this code snippet right here in this code snippet you can see

170
00:14:10,260 --> 00:14:12,960
the function being called on this line.

171
00:14:13,140 --> 00:14:21,270
And if I hit try it then we actually see this image here being converted to black and white to grayscale

172
00:14:21,810 --> 00:14:25,190
and this looks like exactly what we want.

173
00:14:25,230 --> 00:14:32,820
So let's examine this CBT colour function in a bit more detail so we can see it has one two three four

174
00:14:32,850 --> 00:14:33,680
inputs.

175
00:14:33,780 --> 00:14:35,360
What are those.

176
00:14:35,370 --> 00:14:42,310
Well the parameters for this function are fourfold and they're described in detail up here.

177
00:14:42,450 --> 00:14:45,520
So we need a source image which is our input.

178
00:14:45,810 --> 00:14:50,690
Then we get the output here which is the second parameter.

179
00:14:51,050 --> 00:14:58,560
In this case they're calling it DST for maybe destination then we need to supply a colour space.

180
00:14:58,560 --> 00:15:02,400
So there are a number of different conversions you can make.

181
00:15:02,400 --> 00:15:08,970
But the one we're actually interested in is this one here from our GP to grayscale.

182
00:15:09,180 --> 00:15:15,510
The final thing that we can supply are the number of channels in the destination image we can see that

183
00:15:15,510 --> 00:15:17,960
in the code snippet they've supplied a zero here.

184
00:15:18,060 --> 00:15:24,000
And what this means is that if we put in the number zero then the number of channels is derived automatically

185
00:15:24,270 --> 00:15:25,940
from the code.

186
00:15:25,950 --> 00:15:32,630
So if you're converting to grayscale it's gonna be a single channel but you don't have to memorize that.

187
00:15:32,700 --> 00:15:37,980
You can just leave it a zero and open CV will actually be smart enough to figure it out.

188
00:15:38,700 --> 00:15:47,550
So let's copy this line of code here and in our document insert a few more lines here and pasted in.

189
00:15:47,610 --> 00:15:52,930
Now let's modify the code that we've just pasted in to suit our needs.

190
00:15:52,980 --> 00:15:55,150
So what's our source image called.

191
00:15:55,170 --> 00:16:00,400
Well it's image because that's what we're reading from the canvas.

192
00:16:00,420 --> 00:16:02,060
What about the destination.

193
00:16:02,100 --> 00:16:04,470
What should the output be called.

194
00:16:04,510 --> 00:16:06,060
Now that's a little bit more tricky.

195
00:16:06,090 --> 00:16:07,430
So what should our destination be.

196
00:16:07,440 --> 00:16:09,420
What should our output be.

197
00:16:09,420 --> 00:16:10,470
Well how about this.

198
00:16:10,470 --> 00:16:14,010
How would we overwrite or update our image.

199
00:16:14,010 --> 00:16:19,990
So our image is going to be both the input and the output for this function.

200
00:16:20,280 --> 00:16:25,740
And what this will do then is it will update our image to grayscale.

201
00:16:25,920 --> 00:16:35,220
And then when we show it down here we should see a grayscale image now at the moment our canvas is already

202
00:16:35,220 --> 00:16:36,250
black and white.

203
00:16:36,300 --> 00:16:44,070
So in order to test this let's change the line that we're drawing to a kind of a toxic green like some

204
00:16:44,070 --> 00:16:49,890
sour children's candy or the color Psaki racing colors to do that.

205
00:16:49,890 --> 00:16:55,170
We're going to go into our drawing that J S file I'm going to change the line color to a different hex

206
00:16:55,170 --> 00:17:02,160
code the one I'm after is b c f f 0 0 are you look this up.

207
00:17:02,160 --> 00:17:10,260
This is a lime green color so if I save my changes here save my changes in processing and then refresh

208
00:17:10,260 --> 00:17:15,480
my page then I should be able to draw here in a lime green color.

209
00:17:15,720 --> 00:17:19,700
If you're not seeing that just like me then maybe head over to tools.

210
00:17:19,740 --> 00:17:25,530
Clear your browsing data and make sure you're not working with a cached version of your Web site.

211
00:17:25,590 --> 00:17:31,260
If I refresh this again and I draw on my canvas then the lime green color is coming through.

212
00:17:31,260 --> 00:17:33,280
Nicely perfect.

213
00:17:33,570 --> 00:17:36,710
But what about our grayscale conversion.

214
00:17:36,720 --> 00:17:45,240
Well let's hit the button check answer and indeed we see a gray where we previously had the lime green

215
00:17:46,020 --> 00:17:48,660
so the black and white conversion works.

216
00:17:48,660 --> 00:17:55,320
We have gray instead of green no perfect but one of the things that we should maybe do is we should

217
00:17:55,410 --> 00:18:03,300
increase the contrast between the line that we're drawing and the background so similar to how we are

218
00:18:03,300 --> 00:18:06,770
drawing a white line on a black background.

219
00:18:06,810 --> 00:18:15,810
Let's convert our lines to white programmatically the way we can do this with open CV is we use something

220
00:18:15,810 --> 00:18:17,850
called Image threshold.

221
00:18:18,510 --> 00:18:25,360
What this will do is it will make all the pixels above a certain threshold a different color.

222
00:18:25,410 --> 00:18:29,990
The function that we're after is called CV dot threshold.

223
00:18:30,030 --> 00:18:36,450
Once again it takes an input has an output and then we can specify a threshold value so this is the

224
00:18:36,450 --> 00:18:43,380
cutoff value and the maximum value to use for the different threshold types.

225
00:18:43,380 --> 00:18:49,530
Now I know this isn't a great description but what this will do is basically it will convert anything

226
00:18:49,650 --> 00:18:53,100
above this threshold to the maximum value.

227
00:18:53,130 --> 00:18:54,900
That's what this function does.

228
00:18:54,900 --> 00:18:58,320
Let's take a look at the code snippet below for an example.

229
00:18:58,320 --> 00:19:06,350
So here we've got a threshold that's going to convert everything above the value of 1 7 7 to 200.

230
00:19:06,870 --> 00:19:10,630
So if we try it out we get something like this.

231
00:19:10,800 --> 00:19:17,940
If we change the output to 2 5 5 and refresh then the contrast becomes even more marked and this is

232
00:19:17,940 --> 00:19:24,030
perfect for our purposes because what this will allow us to do is make our edges a lot more defined

233
00:19:24,390 --> 00:19:27,440
which is what we need for the next step.

234
00:19:27,510 --> 00:19:34,950
So coming back into V is code let's add the logic to make all the pixels above a certain threshold white

235
00:19:35,280 --> 00:19:37,740
and make the rest black.

236
00:19:37,830 --> 00:19:45,180
We're gonna use the CVR dot threshold function and then we're going to apply for inputs our image and

237
00:19:45,180 --> 00:19:46,200
then our image again.

238
00:19:46,200 --> 00:19:49,470
So the first input was the source image.

239
00:19:49,470 --> 00:19:55,080
Second is our output our destination and then we're gonna select a threshold value.

240
00:19:55,080 --> 00:19:58,730
I want to go for 1 7 5 which is a kind of gray.

241
00:19:59,160 --> 00:20:02,250
I can actually visualize this for you using a keynote right here.

242
00:20:02,580 --> 00:20:09,570
So if I said all three are GV sliders to 1 7 5 then you can see what I'm setting as the threshold.

243
00:20:09,570 --> 00:20:15,570
So anything above this threshold I wanted to be white I wanted to be equal to two hundred and fifty

244
00:20:15,570 --> 00:20:16,480
five.

245
00:20:16,800 --> 00:20:21,530
So that means that two under fifty five is going to be the next argument to my threshold function.

246
00:20:21,660 --> 00:20:29,910
It's gonna be the value that anything above 255 is set to the last thing I need to provide is how the

247
00:20:29,910 --> 00:20:36,150
threshold is calculated and if we look at the documentation him we see there is a number of different

248
00:20:36,150 --> 00:20:40,330
ways but we can use this one here that they're using in the docs as well.

249
00:20:40,410 --> 00:20:44,920
The binary threshold it's either gonna be one or the other.

250
00:20:44,970 --> 00:20:49,330
So let's go with that and add the semicolon at the end.

251
00:20:49,530 --> 00:20:55,230
So if we save our changes and come back to our website here what I'm gonna do next is I want to show

252
00:20:55,230 --> 00:20:57,020
you a side by side comparison.

253
00:20:57,120 --> 00:21:04,650
So when you use a screen capping tool here and I'm going to take a screenshot of these testing canvasses

254
00:21:04,650 --> 00:21:10,870
done here capture them and then what I'll do is I want to refresh.

255
00:21:11,120 --> 00:21:15,640
I want to draw something on the canvas press check answer.

256
00:21:15,950 --> 00:21:23,240
And now if I pull them up here side by side you can tell that previously we had a kind of light gray

257
00:21:23,780 --> 00:21:29,800
and thanks to our threshold logic that we've added all the lines are now stark white.

258
00:21:29,900 --> 00:21:32,060
But that's not the only difference.

259
00:21:32,060 --> 00:21:39,200
The other thing you actually see is that there are more jagged edges on the canvas here than before.

260
00:21:39,500 --> 00:21:45,240
And that's because there's this marked differentiation between the black and white pixels.

261
00:21:45,260 --> 00:21:48,340
So all the pixels that previously had a bit of a shade.

262
00:21:48,440 --> 00:21:51,170
Now have a stock value.

263
00:21:51,170 --> 00:21:58,850
And this is useful for allowing us to figure out what the bounding rectangle should be.

264
00:21:58,850 --> 00:22:04,100
Now in order to calculate the bounding rectangle what we first need to do is work out what the contours

265
00:22:04,100 --> 00:22:11,300
are of what's been drawn on our canvas in order to figure out how to calculate what the contours are

266
00:22:11,960 --> 00:22:20,630
open C.V. once again has a fantastic tutorial and here on the Getting Started page for the contours

267
00:22:21,170 --> 00:22:27,290
we've actually got a wonderful example that they've got right here on how to find the contours in the

268
00:22:27,290 --> 00:22:28,440
first place.

269
00:22:28,670 --> 00:22:30,620
Looking at the codes that are below.

270
00:22:30,680 --> 00:22:35,330
Let's see if we can identify the most important lines in the code.

271
00:22:35,330 --> 00:22:43,640
So we've got fine Quantas right here and find contours has one two three four five inputs and the key

272
00:22:43,640 --> 00:22:46,080
inputs are of course the source.

273
00:22:46,400 --> 00:22:54,020
And then this thing called contours and then this thing called hierarchy and those to come from up here

274
00:22:54,500 --> 00:22:56,470
the first one this is sort of vector.

275
00:22:56,570 --> 00:22:59,570
And the second one is some sort of matrix.

276
00:22:59,660 --> 00:23:02,090
So it's these three lines here.

277
00:23:02,270 --> 00:23:13,520
Let Qantas equals new CV dot match vector let hierarchy equal new CVD up Matt and speed up find contours

278
00:23:13,820 --> 00:23:20,090
that are the lines that we're interested in as you might have guessed from the previous examples the

279
00:23:20,210 --> 00:23:23,930
output of this function here are the contours.

280
00:23:23,960 --> 00:23:29,990
So instead of having some sort of destination here the contours that are being calculated by find contours

281
00:23:30,560 --> 00:23:38,210
are the second parameter here and the hierarchy is the information about the image topology.

282
00:23:38,210 --> 00:23:42,560
The next thing that we need to specify in this function are a mode and a method.

283
00:23:42,560 --> 00:23:48,230
So some sort of way of retrieving the contours and some sort of approximation method.

284
00:23:48,590 --> 00:23:53,610
And we're not gonna dive into detail here as to like the pros and cons of the different methods.

285
00:23:53,810 --> 00:23:56,430
The example code works just fine.

286
00:23:56,510 --> 00:24:03,160
And to prove it we can hit the tried button here and it even draws what the controls are of this image.

287
00:24:03,680 --> 00:24:10,770
So let's copy these three lines of code and come back into v as code and paste them in.

288
00:24:10,850 --> 00:24:18,350
Now of course we'll have to modify them so our source of course is not called as our C it's called image

289
00:24:19,580 --> 00:24:22,690
but the other two we can live as they are.

290
00:24:22,770 --> 00:24:28,670
One thing I want to do of course is a format that the documents so we get this nice clean indentation

291
00:24:29,150 --> 00:24:37,460
and that puts us in a great place for the next step namely calculating the bounding rectangle and actually

292
00:24:37,460 --> 00:24:40,580
want to leave this as a challenge to you.

293
00:24:40,580 --> 00:24:48,170
So what I'd like you to do is find the pages in the open CV documentation that allow you to figure out

294
00:24:48,590 --> 00:24:54,770
how to write the code in order to calculate the bounding rectangle for whatever was drawn on the canvas

295
00:24:55,670 --> 00:25:02,290
and then what I'd like you to do is use this bounding rectangle to crop the image.

296
00:25:02,540 --> 00:25:08,150
In fact in order to complete this challenge you actually only have to write three lines of code but

297
00:25:08,450 --> 00:25:14,120
you're gonna have to find out for yourself which two lines of code are going to create the bounding

298
00:25:14,120 --> 00:25:19,400
rectangle and then what line of code will help you crop the image.

299
00:25:19,400 --> 00:25:24,410
Now mind you you don't actually have to draw the bounding rectangle on the canvas like I've got here

300
00:25:24,410 --> 00:25:25,450
on the slide.

301
00:25:25,550 --> 00:25:30,350
All you need to do is crop the image to whatever part of the canvas.

302
00:25:30,440 --> 00:25:38,030
Our lines were drawn and I'll give you a few seconds to pause the video hunt around in the open CV documentation

303
00:25:38,390 --> 00:25:39,830
and give this challenge a go

304
00:25:43,300 --> 00:25:44,310
ready.

305
00:25:44,320 --> 00:25:45,850
Here's the solution.

306
00:25:45,850 --> 00:25:49,400
You're gonna have to combine two of open CPS functions.

307
00:25:49,690 --> 00:25:59,380
The first one is this bounding rect method and the second one is this region of interest method with

308
00:25:59,380 --> 00:26:03,580
those two functions in hand you can complete the challenge.

309
00:26:03,700 --> 00:26:07,310
So how does this bounding rectangle work.

310
00:26:07,330 --> 00:26:10,950
This is part of the contours tutorial by the way.

311
00:26:11,290 --> 00:26:17,140
And this is the reason why we worked out the contours first in the code snippet.

312
00:26:17,230 --> 00:26:22,530
You can see that in order to figure out what the bounding rectangle is you need these two lines of code.

313
00:26:22,660 --> 00:26:28,900
You need to get some contours from the controls object and then you're gonna use these contours which

314
00:26:28,900 --> 00:26:33,940
are abbreviated here with S.A. to calculate the bounding rectangle.

315
00:26:33,940 --> 00:26:34,470
Right here.

316
00:26:34,750 --> 00:26:42,850
So these contours which is a subset of all the contours feed into the bounding rect method and allow

317
00:26:42,850 --> 00:26:45,660
you to work out the bounding rectangle.

318
00:26:45,670 --> 00:26:47,670
Now what about the cropping.

319
00:26:47,710 --> 00:26:50,190
Well this is where are you going to use this bounding rectangle.

320
00:26:50,200 --> 00:26:51,020
Right.

321
00:26:51,160 --> 00:26:58,780
If we come over here to the tutorial on the basic operations and open CV we can see that the region

322
00:26:58,780 --> 00:27:04,170
of interest here can be set using a rectangle.

323
00:27:04,210 --> 00:27:05,040
Right.

324
00:27:05,140 --> 00:27:13,090
So we can crop the image using the bounding rectangle that we've calculated from the controls.

325
00:27:13,090 --> 00:27:14,020
That's the trick.

326
00:27:14,620 --> 00:27:17,550
So let's implement this in V.S. code.

327
00:27:17,570 --> 00:27:25,090
We're gonna use the contours which are the output from fine contours and we're going to store the contrast

328
00:27:25,090 --> 00:27:29,930
that we're interested in this subset in a variable called S.A..

329
00:27:30,550 --> 00:27:34,610
So it'll be contour start to get zero.

330
00:27:34,630 --> 00:27:41,040
Just like it was outlined in the documentation and then we're to create a rectangle using these contours.

331
00:27:41,140 --> 00:27:50,700
So we see the bounding rect parentheses S.A. and then we're gonna use this rectangle to crop our image.

332
00:27:50,800 --> 00:27:52,780
So let's update our image right.

333
00:27:52,780 --> 00:27:55,160
This is what we usually do.

334
00:27:55,360 --> 00:28:00,390
We are going to set image equal to image dot Ah.

335
00:28:00,610 --> 00:28:03,290
Oh I reached of interest.

336
00:28:03,310 --> 00:28:10,210
And then as an input the region of interest method needs a rectangle and that's it.

337
00:28:10,210 --> 00:28:17,200
Those are the three lines of code that allow us to crop our image and just to test it out and see it

338
00:28:17,200 --> 00:28:17,710
in action.

339
00:28:17,720 --> 00:28:20,060
I'm when I save the work here.

340
00:28:20,110 --> 00:28:26,990
Come over here refresh and then what I'll do is I'll draw something maybe in this corner right here.

341
00:28:27,100 --> 00:28:27,710
Click check.

342
00:28:27,720 --> 00:28:33,670
Answer and you can see that all the parts of the canvas that were not drawn on were cropped.

343
00:28:33,670 --> 00:28:41,470
If I draw something very large then it's going to crop very very little around the edges so the bounds

344
00:28:41,650 --> 00:28:46,090
of the rectangle are just where our lines touch the edge.

345
00:28:46,090 --> 00:28:47,430
So this is fantastic right.

346
00:28:47,440 --> 00:28:52,160
Any shape you want you can draw and it'll crop it to the edges.

347
00:28:52,240 --> 00:28:58,050
So I hope you enjoy that challenge and this brings us almost to the end of this lesson.

348
00:28:58,090 --> 00:29:03,750
There's just a couple more things to do and that's doing a little bit of cleanup actually.

349
00:29:03,910 --> 00:29:06,940
So I'm not gonna leave this in this lime green here.

350
00:29:07,060 --> 00:29:09,660
I'm going to set my line color back to white.

351
00:29:09,850 --> 00:29:17,440
So I want to come here in the drawing not just file and change this back to six FS which is the hex

352
00:29:17,440 --> 00:29:23,740
code for white and then what we're gonna do is we do a bit of cleanup on the open CV front.

353
00:29:24,280 --> 00:29:30,820
So if you've had a look at the basic operations on images page then what you see actually are a lot

354
00:29:30,820 --> 00:29:31,940
of these notes.

355
00:29:31,960 --> 00:29:37,420
Don't forget to delete CV mat when you don't want to use it anymore for example and this is almost below

356
00:29:37,420 --> 00:29:39,030
every example.

357
00:29:39,100 --> 00:29:47,380
So a lot of these objects here like Matt Matt Vec S.A. are being deleted at the end.

358
00:29:47,380 --> 00:29:49,570
So we're gonna go with this recommendation as well.

359
00:29:49,660 --> 00:29:55,390
We're gonna delete the open see the objects that we're not using anymore in order to free up memory

360
00:29:55,840 --> 00:30:00,490
and kind of comply with the recommendations in the documentation.

361
00:30:00,550 --> 00:30:02,420
So what are the things that we should delete.

362
00:30:02,710 --> 00:30:09,430
We want to add a little comment here that reads clean up and then I'm going to delete my image so I'm

363
00:30:09,430 --> 00:30:16,210
not going to need my image anymore my image object and I'm gonna do that with image not delete parentheses

364
00:30:16,240 --> 00:30:22,300
and a colon but there's a couple of other objects as well namely these controls here.

365
00:30:22,300 --> 00:30:28,630
These controls here and the hierarchy so I'm gonna delete those as well.

366
00:30:28,630 --> 00:30:43,210
Contours don't delete S.A. don't delete and hierarchy delete delete and that's it.

367
00:30:43,210 --> 00:30:47,740
Now we're ready to tackle part two of data processing.

368
00:30:47,830 --> 00:30:48,940
I'll see you in the next lesson.